Frame synchronous packet switching for high-definition multimedia interface (hdmi) video transitions

ABSTRACT

An apparatus for use in a high-definition media interface (HDMI) source device includes an HDMI interface for transmitting video data and metadata to a sink device. The apparatus is configured to encode the metadata in an auxiliary video information (AVI) information frame (InfoFrame). The apparatus is further configured to transmit the AVI InfoFrame during a frame synchronous transmission window (FSTW) of the video data, wherein the FSTW begins during a video blanking interval (VBI) of the video data, on a first video blank pixel that immediately follows a last active video pixel of a preceding video frame or video field and ends a predetermined number of video lines after a start of the VBI.

RELATED APPLICATION

This application is a continuation of and claims the benefit of priorityto U.S. application Ser. No. 15/982,838, filed May 17, 2018, whichapplication is incorporated by reference as if reproduced herein andmade a part hereof in its entirety, and the benefit of priority of whichis claimed herein.

TECHNICAL FIELD

This application concerns sending and receiving units that employ ahigh-definition multimedia interface (HDMI) and in particular to HDMIsending and receiving units implementing frame synchronous transitionsamong high dynamic range (HDR) and standard dynamic range (SDR) videocontent.

BACKGROUND

The high-definition multimedia interface (HDMI) is a popular interfacefor transmitting high-speed baseband digital video and associated audiosignals for presentation on an HDMI-capable device. Recently, highdynamic range (HDR) video display devices have become available, andvideo sources, such as digital versatile disc (DVD) players, televisionbroadcasts, and on-line streaming services, now provide HDR content. HDRdisplays that receive HDR content provide higher brightness levels andmay also provide darker black levels and improved color rendering ascompared to standard dynamic range (SDR). SDR video refers to a dynamicrange of between zero and 300 nits (cd/m²). Recently, display deviceshaving dynamic ranges up to 10000 nits or greater have become available.These display devices are referred to as HDR displays. In order toaccommodate these HDR displays and the corresponding HDR sources, videointerfaces, including HDMI, have been adapted to transport both pixeldata and SDR or HDR metadata over the interface.

Metadata for SDR video data is sent over the HDMI interface usingauxiliary video information (AVI) information frames (InfoFrames).Currently, there are two types of HDR metadata, static HDR (S-HDR)metadata which is sent using DRange and Mastering (DRAM) InfoFrames, anddynamic HDR metadata which is sent using HDR Dynamic Metadata Extended(HDR DME) InfoFrames. S-HDR metadata is applied to an entire programwhile dynamic HDR metadata may change more frequently, typically over asequence of several frames but could change frame to frame. The metadatain the DRAM InfoFrames and HDR DME InfoFrames augments the metadata inthe AVI InfoFrames.

A source processing an HDR signal may be coupled to a sink (e.g.,display) configured to display only SDR video or SDR video and one orboth of S-HDR video or dynamic HDR video. When the sink does not supportdynamic HDR, the source may convert the dynamic HDR video data to S-HDRvideo data or SDR video data before sending the video data to the sink.When the sink does not support S-HDR video or dynamic HDR video, thesource may convert both S-HDR video data and dynamic HDR video data toSDR video data before sending the video data to the sink. A sink that iscapable of displaying dynamic HDR video receives the video data over theHDMI interface using the HDR DME InfoFrame in a frame-synchronous mannerso that the metadata is applied to the frame occurring immediately afterthe metadata is received.

To implement the frame-synchronous switching of the dynamic HDR metadatacarried in the HDR DME InfoFrame, HDMI 2.1 defines a frame accuratepacket area (FAPA) in the vertical blanking area of the video signal andspecifies that HDR DME InfoFrames are to be sent during the FAPA period.HDMI 2.1 also specifies that AVI InfoFrames and DRAM InfoFrames are tobe sent in a frame-synchronous manner, but HDMI 2.1 does not requirethat these InfoFrames be sent during any particular period within avideo frame. Therefore, considering the timing requirements specifiedfor transmission of InfoFrames, the timing of the HDR DME InfoFrame, isprecisely specified to be transmitted during the FAPA period. The AVIInfoFrame and the DRAM InfoFrame are required to be frame-synchronous,but a specific time period for transmission is not specified.

SUMMARY

Various examples are now described to introduce a selection of conceptsin a simplified form that are further described below in the detaileddescription. The Summary is not intended to identify key or essentialfeatures of the claimed subject matter, nor is it intended to be used tolimit the scope of the claimed subject matter.

According to one aspect of the present disclosure, an apparatus for usein a source device for transmitting and receiving data using a highdefinition media interface (HDMI), the apparatus comprises an HDMIinterface for transmitting data to and receiving data from a sinkdevice; a memory holding executable code; a processor, coupled to thememory and to the HDMI interface, the processor configured by theexecutable code to: receive video data and metadata for transmission tothe sink device; encode the metadata in an auxiliary video information(AVI) information frame (InfoFrame); and transmit the AVI InfoFrameduring a frame synchronous transmission window (FSTW) of the video data,wherein the FSTW begins during a video blanking interval (VBI) of thevideo data, on a first video blank pixel that immediately follows a lastactive video pixel of a preceding video frame or video field and ends apredetermined number of video lines after a start of the VBI.

Optionally, in the preceding aspect, a further implementation of theaspect includes, the received video data including standard dynamicrange (SDR) video data and the metadata is metadata for the SDR videodata.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes, the received metadata including metadata for astatic high dynamic range (S-HDR) video sequence wherein the processoris configured by the executable code to: encode the metadata for theS-HDR video sequence in the AVI InfoFrame and in a DRange and Mastering(DRAM) InfoFrame; and transmit the AVI InfoFrame and the DRAM InfoFrameduring the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes, the received metadata including metadata for adynamic high dynamic range (HDR) video sequence wherein the processor isconfigured by the executable code to: encode the metadata for thedynamic HDR video sequence in the AVI InfoFrame and in a HDR dynamicmetadata extended (HDR DME) InfoFrame; and transmit the AVI InfoFrameand the HDR DME InfoFrame during the FSTW.

According to another aspect of the present disclosure, an apparatus foruse in a sink device for receiving data using a high definition mediainterface (HDMI), the apparatus comprises: an HDMI interface forreceiving data from a source device; a memory holding executable code; aprocessor, coupled to the memory and to the HDMI interface, theprocessor configured by the executable code to: receive a video sequencefrom the source device, the video sequence including a plurality ofvideo fields or video frames, each video field or video frame includingan active video interval and a vertical blanking interval (VBI); extractan auxiliary video information (AVI) information frame (InfoFrame)including metadata for the video sequence from a frame synchronoustransmission window (FSTW) of the VBI of at least one of the fields orframes of the video sequence, wherein the FSTW begins during the VBI ona first video blank pixel that immediately follows a last active videopixel of a preceding video field or video frame and ends a predeterminednumber of video lines after a start of the VBI; extract the metadatafrom the AVI InfoFrame; and apply the extracted metadata to video datain the active video interval of the video field or video framecontaining the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes the received video sequence having a static highdynamic range (S-HDR) video sequence wherein the processor is configuredby the executable code to: extract a DRange and Mastering (DRAM)InfoFrame from the FSTW of the VBI of the at least one field or frame ofthe video sequence; extract further metadata from the DRAM InfoFrame;and apply the extracted metadata and the further metadata to the videodata in the active video interval of the video field or video framecontaining the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes the received video sequence having a high dynamicrange (HDR) video sequence wherein the processor is configured by theexecutable code to: extract an HDR dynamic metadata extended (HDR DME)InfoFrame from the FSTW of the VBI of the at least one field or frame ofthe video sequence; extract further metadata from the HDR DME InfoFrame;and apply the extracted metadata and the further metadata to the videodata in the active video interval of the video field or video framecontaining the FSTW.

According to another aspect of the present disclosure, a method fortransmitting data from a source device to a sink device uses a highdefinition media interface (HDMI) and comprises: receiving video dataand metadata for transmission to the sink device; encoding the metadatain an auxiliary video information (AVI) InfoFrame; and transmitting theAVI InfoFrame during a frame synchronous transmission window (FSTW) ofthe video data, wherein the FSTW begins during a video blanking interval(VBI) of the video data, on a first video blank pixel that immediatelyfollows a last active video pixel of a preceding video frame or videofield and ends a predetermined number of video lines after a start ofthe VBI.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes receiving standard dynamic range (SDR) video dataand the metadata is metadata for the SDR video data.

Optionally, in any of the preceding aspects, in a further implementationof the aspect, receiving the video data and metadata includes receivinga static high dynamic range (S-HDR) video sequence and metadata for theS-HDR video sequence; encoding the metadata includes encoding themetadata for the S-HDR video sequence in the AVI InfoFrame and in aDRange and Mastering (DRAM) InfoFrame; and transmitting the AVIInfoFrame includes transmitting the AVI InfoFrame and the DRAM InfoFrameduring the FSTW.

Optionally, in any of the preceding aspects, in a further implementationof the aspect, receiving the video data and metadata includes receivinga dynamic high dynamic range (HDR) video sequence and metadata for thedynamic HDR video sequence; encoding the metadata includes encoding themetadata for the dynamic HDR video sequence in the AVI InfoFrame and ina HDR dynamic metadata extended (HDR DME) InfoFrame; and transmittingthe AVI InfoFrame includes transmitting the AVI InfoFrame and the HDRDME InfoFrame during the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes receiving a video sequence from the source device,the video sequence including a plurality of video fields or videoframes, each video field or video frame including an active videointerval and a vertical blanking interval (VBI); extracting an auxiliaryvideo information (AVI) information frame (InfoFrame) including metadatafor the video sequence from a frame synchronous transmission window(FSTW) of the VBI of at least one of the fields or frames of the videosequence, wherein the FSTW begins during the VBI on a first video blankpixel that immediately follows a last active video pixel of a precedingvideo field or video frame and ends a predetermined number of videolines after a start of the VBI; extracting the metadata from the AVIInfoFrame; and applying the extracted metadata to video data in theactive video interval of the video field or video frame containing theFSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes extracting a DRange and Mastering (DRAM) InfoFramefrom the FSTW of the VBI of the at least one field or frame of the videosequence; extracting further metadata from the DRAM InfoFrame; andapplying the further metadata to the S-HDR video data in the activevideo interval of the video field or video frame containing the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes extracting an HDR dynamic metadata extended (HDRDME) InfoFrame from the FSTW of the VBI of the at least one field orframe of the video sequence; extracting further metadata from the HDRDME InfoFrame; and applying the further metadata to the dynamic HDRvideo data in the active video interval of the video field or videoframe containing the FSTW.

According to another aspect of the present disclosure, acomputer-readable medium includes program instructions for execution bya processor to configure the processor to transmit data from a sourcedevice to a sink device using a high definition media interface (HDMI),the program instructions configuring the processor to: receive videodata and metadata for transmission to the sink device; encode themetadata in an auxiliary video information (AVI) information frame(InfoFrame); and configure the AVI InfoFrame for transmission during aframe synchronous transmission window (FSTW) of the video data, whereinthe FSTW begins during a video blanking interval (VBI) of the videodata, on a first video blank pixel that immediately follows a lastactive video pixel of a preceding video frame or video field and ends apredetermined number of video lines after a start of the VBI.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes program instructions to configure the processor to:receive standard dynamic range (SDR) video data and metadata for the SDRvideo data; and encode the metadata for the SDR video data in the AVIInfoFrame.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes program instructions to configure the processor to:receive, as the video data and metadata, a static high dynamic range(S-HDR) video sequence and metadata for the S-HDR video sequence; encodethe metadata for the S-HDR video sequence in the AVI InfoFrame and in aDRange and Mastering (DRAM) InfoFrame; and configure the AVI InfoFrameand the DRAM InfoFrame for transmission during the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes program instructions to configure the processor to:receive, as the video data and metadata, a dynamic high dynamic range(HDR) video sequence and metadata for the dynamic HDR video sequence;encode the metadata for the dynamic HDR video sequence in the AVIInfoFrame and in a HDR dynamic metadata extended (HDR DME) InfoFrame;and configure the AVI InfoFrame and the HDR DME InfoFrame fortransmission during the FSTW.

According to yet another aspect of the present disclosure, acomputer-readable medium includes program instructions for execution bya processor to configure the processor in a sink device to receive datafrom a source device using a high definition media interface (HDMI), theprogram instructions configuring the processor to: receive a videosequence from the source device, the video sequence including aplurality of video fields or video frames, each video field or videoframe including an active video interval and a vertical blankinginterval (VBI); extract an auxiliary video information (AVI) informationframe (InfoFrame) including metadata for the video sequence from a framesynchronous transmission window (FSTW) of the VBI of at least one of thefields or frames of the video sequence, wherein the FSTW begins duringthe VBI on a first video blank pixel that immediately follows a lastactive video pixel of a preceding video field or video frame and ends apredetermined number of video lines after a start of the VBI; extractthe metadata from the AVI InfoFrame; and apply the extracted metadata tovideo data in the active video interval of the video field or videoframe containing the FSTW.

Optionally, in any of the preceding aspects, a further implementation ofthe aspect includes program instructions to configure the processor to:receive, as the video sequence, a static high dynamic range (S-HDR)video sequence and the method further comprises: extracting a DRange andMastering (DRAM) InfoFrame from the FSTW of the VBI of the at least onefield or frame of the video sequence; extracting further metadata fromthe DRAM InfoFrame; and applying the further metadata to the video datain the active video interval of the video field or video framecontaining the FSTW.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of an HDMI sending unit and receiving unitincluding transition minimized differential signaling (TMDS) laneschannels according to an example embodiment.

FIG. 1B is a block diagram of an HDMI sending unit and receiving unitincluding fixed rate link (FRL) lanes according to an exampleembodiment.

FIG. 2 is a block diagram of an HDMI source device according to anexample embodiment.

FIG. 3 is a block diagram of an HDMI sink device according to an exampleembodiment.

FIG. 4 is a timing diagram showing a sequence of video fields/framesincluding frame synchronous transmission windows (FSTWs) according to anexample embodiment.

FIG. 5 is a timing diagram showing a single video field/frame accordingto an example embodiment.

FIG. 6 is a timing diagram showing a stitched linear video stream havingstatic high dynamic range (S-HDR) and standard dynamic range (SDR) videosequences.

FIG. 7 is a timing diagram showing a stitched linear video stream havingincluding SDR, S-HDR, and dynamic HDR video sequences.

FIG. 8 is a timing diagram showing a stitched linear video streamincluding SDR, S-HDR, and dynamic HDR video sequences according toexample embodiments.

FIG. 9A is a flowchart diagram useful for describing the operation of asource device according to an example embodiment.

FIG. 9B is a flowchart diagram useful for describing the operation of asink device according to an example embodiment.

FIG. 10 is a state diagram useful for describing differences among theexample embodiments, legacy HDMI, and HDMI 2.1.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings that form a part hereof, and in which is shown by way ofillustration specific embodiments which may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the subject matter, and it is to be understoodthat other embodiments may be utilized and that structural, logical, andelectrical changes may be made without departing from the scope of thepresent subject matter. The following description of example embodimentsis, therefore, not to be taken in a limited sense, and the scope of thepresent subject matter is defined by the appended claims.

The functions or algorithms described herein may be implemented insoftware in one embodiment. The software may consist ofcomputer-executable instructions stored on computer-readable media orcomputer-readable storage device such as one or more non-transitorymemories or other type of hardware based storage devices, either localor networked. Further, such functions correspond to modules, which maybe software, hardware, firmware, or any combination thereof. Multiplefunctions may be performed in one or more modules as desired, and theembodiments described are merely examples. The software may be executedon processing circuitry that may include a single core microprocessor,multi-core microprocessor, digital signal processor, applicationspecific integrated circuit (ASIC), field programmable gate array(FPGA), or other type of data processing circuitry operating on acomputer system, such as a personal computer, server or other computersystem, turning such computer system into a specifically programmedmachine.

In many existing systems, video information originates from a singlesource such as a digital versatile disk (DVD) player or a televisiontuner. These sources typically provide video data with a uniform dynamicrange and may provide either SDR data or S-HDR data. To display videodata from these sources, the HDMI interface provides for S-HDR metadatasignaling (e.g., AVI InfoFrames and DRAM InfoFrames) and SDR signaling(e.g., AVI InfoFrames).

S-HDR signaling works well when the video data changes between HDR andSDR infrequently (e.g., when an S-HDR disk is inserted in the DVDplayer). Increasingly, however, video data is provided in a streamingformat in which disparate video segments are stitched together into asingle stream. Some segments may be SDR segments while others are HDRsegments. As described below with reference to FIG. 6, there may be arelatively short period after the switch between displaying the S-HDRand SDR signals in which the display device produces a slightlydistorted image. This distortion occurs, for example, on switchingbetween television programs when SDR signals are processed usingmetadata intended for displaying S-HDR signals or vice versa. Becausethis distortion is relatively minor, infrequent, and short in duration,it has generally been ignored.

More recently, different types of HDR video data may be provided in asingle scene or for a single frame. For example, in a relatively darkscene, the range of luminance values may be significantly less than thefull range of the HDR signal. For example, a 10-bit luminance signal mayhave values bounded by 0-255, the range of an 8-bit video signal. Inthis instance, an opto-electric transfer function (OETF) andcorresponding electro-optical transfer function (EOTF) may be applied sothat the image data in the scene may be mapped into the 10-bit range ofthe luminance signal, reducing quantization distortion in the reproducedimage. These signals are dynamic HDR signals that may use HDR DMEInfoFrames to send the EOTF to the sink device.

Because the dynamic HDR video signals having HDR DME may change on aframe-by-frame basis, the HDR DME InfoFrames are processed withframe-synchronous timing to ensure proper display of the HDR video data.The embodiments described below also send AVI InfoFrames and DRAMInfoFrames in a frame-synchronous transmission window (FSTW). The FSTW,which has the same timing as FAPA with location start 0 (FAPA0), startson the first video blank pixel that immediately follows the last activevideo pixel of a video frame/field and ends FAPA_end lines prior to thestart of the next active region (as described in section 10.10.1.1 ofthe High-Definition Multimedia Interface Specification Version 2.1).Briefly, FAPA_end may be one-half the number of lines in the VBI orless, depending on the number of lines in the VBI. The FSTW is used bysink devices compatible with dynamic HDR video and has timing thatcorresponds to the FAPA. Sending the AVI InfoFrames and DRAM InfoFramesas well as HDR DME InfoFrames during the FSTW reduces image distortionthat may occur on switching among SDR, S-HDR and dynamic HDR videoformats. As used herein, FSTW is identical to FAPA0.

FIG. 1A is a block diagram of an HDMI system 100 having a sending unit110 and receiving unit 150 and including transition-minimizeddifferential signaling (TMDS) channels according to an exampleembodiment. In the system 100, and HDMI sending unit 110 is coupled toan HDMI receiving unit 150 by an HDMI cable 140. The HDMI sending unit110 is a component of an HDMI source device (not shown in FIG. 1A) andthe HDMI receiving unit 150 is a component of an HDMI sink device (notshown in FIG. 1A). The sending unit 110 includes an HDMI transmitter 112that receives video data 114, audio data 116, and control and statusdata 118. The HDMI cable 140 connecting the HDMI sending unit 110 andreceiving unit 150 includes three TMDS channels 120, 122, and 124; aTMDS clock channel 126; a display data channel (DDC) 128, a consumerelectronics control (CEC) channel 130; and a hot plug detect (HPD)channel 132. The HDMI receiving unit 150 includes an HDMI receiver 152that receives differentially encoded data via the TMDS channels 120,122, and 124 at times determined by the TMDS clock channel 126 anddecodes the received data to provide video data 154, audio data 156, andcontrol/status data 158.

The TMDS channels 120, 122, and 124 allow the source device to transmitvideo and audio data 154, 156 to the sink device at rates up to 6gigabits per second (Gbps) using differential signals synchronized bythe clock signal transmitted through the TMDS clock channel 126. Theaudio data 156 may be encoded in data islands, described below, that aretransmitted in the vertical and horizontal blanking intervals of thetransmitted video data 154.

The DDC 128 is a serial channel that includes a serial data (SDA)conductor (not separately shown) and a serial clock (SCL) conductor (notseparately shown). The DDC 128 is used to send/receive control databetween the sending unit 110 and the receiving unit 150. For example,the sending unit 110 may use the DDC 128 to read enhanced extendeddisplay identification data (E-EDID), such as a vendor-specific datablock (VSDB) from the receiving unit 150. For this operation, thereceiving unit 150 may include a read only memory (ROM) (not shown) thatstores the E-EDID of the HDMI receiving unit 150.

The sending unit 110 uses the HPD line to sense that the sink device iscoupled to the cable 140 and is powered on. Responsive to the HPD linehaving a positive DC bias potential, the sending unit 110 reads theE-EDID data via the DDC 128 to determine the capabilities of thereceiving unit 150. The CEC channel 130 allows users to control devicesconnected by the HDMI cable 140 using a single remote control device(not shown). As described below, the E-EDID may include informationabout the HDR capabilities of the sink device, for example, whether thesink device supports S-HDR and/or dynamic HDR.

FIG. 1B is a block diagram of an HDMI sending unit 162 and receivingunit 164 connected by an HDMI cable 182 including fixed rate link (FRL)lanes (channels) 166, 168, 170, and 172 according to an exampleembodiment. This embodiment differs from the embodiment shown in FIG. 1Ain that the three TMDS channels 120, 122, and 124 have been replaced bythree fixed rate link (FRL) lanes 166, 168 and 170. In addition, theTMDS clock channel 126 has been replaced by a fourth FRL lane 172. TheFRL lanes employ 16b18b encoding, and each lane can support data ratesup to 12 Gbps, providing a bandwidth of up to 48 Gbps when all fourlanes are used. The clock signal is encoded in the FRL data, so aseparate clock channel is not needed. The HDMI system is backwardscompatible, so the FRL lanes 166, 168, 170, and 172 can support threeTMDS data channels and a TMDS clock channel as shown in FIG. 1A. Theremaining components of the cable 182—the DDC/SDA/SCL channel 174, CEC176, and HPD channel 180—operate in the same way as the correspondingchannels 128, 130, and 132 shown in FIG. 1A.

FIG. 2 is a block diagram of an example HDMI source device 200 accordingto an example embodiment. The example HDMI source device 200 is able toprovide SDR, S-HDR, and dynamic HDR video sequences and correspondingmetadata to a compatible sink device 300 for frame synchronousprocessing. In addition to the HDMI sending unit 210 and HDMI connector222, the source device 200 includes a processor 202, a memory 204, adisplay controller 206, a network interface 208, a DVD interface 220, anaudio video decoder 214, InfoFrame processing circuitry 216, andmetadata acquisition circuitry 218. The HDMI sending unit 210 includesan HDMI transmitter 211 and a communication interface 212. The examplesource device 200 may be a DVD player having a network interface coupledto receive streaming video data. The device 200 may also receivecompressed audio and video data at an input to the audio video decoder214 and metadata via the metadata acquisition circuitry 218.

The processor 202 controls the operation of other components of the HDMIsource device 200. The memory 204 holds data and instructions for theprocessor 202. The processor 202 may operate the display controller 206to control a display panel (not shown) used to control the operation ofthe HDMI source device 200. The display controller 206 may alsointerface with an input device such as a touchscreen and/or keypad (notshown) to allow a user to input data for controlling the HDMI sourcedevice 200. The processor 202 may also control the network interface 208to allow the source device 200 to access media content from a network(e.g., the Internet) via a browser or a video streaming application. Asdescribed above, this media content may be streaming video including SDRsegments, S-HDR segments, and/or dynamic HDR segments. The communicationinterface 212 of the HDMI sending unit 210 is controlled by theprocessor 202 to communicate with the sink device (described below withreference to FIG. 3) via the DDC/SDA/SCL channel 174 of the HDMIinterface. shown in FIG. 1B. The processor 202 uses this interface tosend commands and data to, and to receive commands and data from, thesink device via the communication interface 212. For example, the sourcedevice 200 may use the communication interface 212 to read the E-EDID ofthe sink device to determine whether the sink device is able to processdynamic HDR video data.

In the example source device 200, compressed video and audio data fromthe DVD interface 220 and/or the network interface 208 are provided tothe audio video decoder 214. The decoder 214 may include a motionpicture experts group (MPEG) decoder such as an H.222/H.262 (MPEG2),H.264 advanced video coding (AVC), and/or H.265 high efficiency videocoding (HEVC) decoder. The decoder 214 generates baseband video andaudio data from the encoded data provided by the network interface 208,DVD interface 220, or provided directly to the AV decoder 214 asindicated in FIG. 2. AV decoder 214 provides the baseband audio andvideo data to the HDMI sending unit 210. The audio and video data areapplied to the HDMI transmitter 211 and are sent through the HDMI TMDSchannels 120, 122, 124, and 126 or through the FRL lanes 166, 168, 170,and 172, described above with reference to FIGS. 1A and 1B, to an HDMIreceiving unit of the sink device. As described above, the video datamay be sent during the active region of the video signal and the audiodata may be send in data islands during the vertical and/or horizontalblanking intervals of the video signal.

When the encoded video stream includes high dynamic range video data,the audio/video decoder 214 extracts the HDR metadata (e.g., DRAM and/orHDR DME) from the encoded video data and provides it to the HDMI sendingunit 210 to be included in data islands to be transmitted inside oroutside of frame synchronous transmission windows (FSTWs) of the videodata sent to the HDMI receiving unit. For video data provided directlyto the audio video decoder 214, any associated HDR metadata may beprovided to the metadata acquisition circuitry 218. This metadata may beprovided to the InfoFrame processing circuitry 216 to be included in thedata islands transmitted by the HDMI transmitter 211.

If the sink device 300 (FIG. 3) supports frame-synchronous processing,then the example InfoFrame processing circuitry 216 formats the metadatasent by the HDMI transmitter 211 so that the AVI InfoFrames, DRAMInfoFrames, and HDR DME InfoFrames are all sent in data islands duringFSTWs of the video signal. Alternatively, when the source device 200determines that the sink device 300 does not support dynamic HDR, theInfoFrame processing circuitry 216 does not format the HDR DME fortransmission to the sink device 300. The InfoFrames for sink devicesthat do not support dynamic HDR may be sent in data islands of the sameportion of the vertical blanking interval as the FSTW (i.e., starting atthe first blank pixel that immediately follows the last active videopixel of a video frame/field and ending FAPA_end lines prior to thestart of the next active region). Because the sink device does notsupport dynamic HDR and, thus, does not support FSTWs, metadata sent inthese data islands will not receive frame synchronous processing.

FIG. 3 is a block diagram of a sink device 300 according to an exampleembodiment. The sink device 300 is able to process video streamsincluding SDR, S-HDR and dynamic HDR video sequences andframe-synchronously process metadata for all of the video sequences. Theexample sink device 300 includes a processor 302, memory 304, displaycontroller 306, audio processing circuitry 318, video processingcircuitry 316, InfoFrame processing circuitry 314, and an HDMI receivingunit 310 including HDMI receiver 311 and communication interface 312.The HDMI receiving unit 310 is coupled to an HDMI connector 308.

The processor 302 controls the operation of other components of the HDMIsink device 300. The memory 304 holds data and instructions for theprocessor 302. The processor 302 may operate the display controller 306to control a display panel (not shown) used to control the operation ofthe HDMI sink device 300. The controller 306 may also interface with aninput device such as a touchscreen and/or keypad (not shown) to allow auser to input data for controlling the HDMI sink device 300. The sinkdevice 300 receives audio and video data via the TMDS channels 120, 122,124 and 126 or FRL lanes 166, 168, 170 and 172, described above withreference to FIGS. 1A and 1B. The example sink device 300 extracts theAVI InfoFrames, DRAM InfoFrames and/or HDR DME InfoFrames containing theSDR and HDR metadata from data islands of the FSTW region of VBI of thevideo signals and provides the metadata to the video processingcircuitry 316.

The HDMI receiving unit 310 extracts audio data from the data islands inthe horizontal and vertical blanking intervals of the video signaloutside of the FSTW and provides the audio data to the audio processingcircuitry 318. The audio data generated by the audio processingcircuitry 318 and the video data generated by the video processingcircuitry 316 are provided to a presentation device including a monitor(not shown) and a sound system (not shown).

Each of the memories 204 and 304 may include volatile memory and/ornon-volatile memory. The non-volatile memory may include removablestorage and non-removable storage. Computer storage includes randomaccess memory (RAM), read-only memory (ROM), erasable programmableread-only memory (EPROM) and electrically erasable programmableread-only memory (EEPROM), flash memory or other memory technologies,compact disc read-only memory (CD ROM), Digital Versatile Disks (DVD) orother optical disk storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumcapable of storing computer-readable instructions.

The various processing devices and circuits shown in FIGS. 2 and 3 mayemploy computer-readable instructions stored on a computer-readablemedium that are executable by the processor 202, audio/video decoder214, InfoFrame processing circuitry 216, and/or metadata acquisitioncircuitry 218 of the source device 200 or the processor 302, InfoFrameprocessing circuitry 314, video processing circuitry 316, and/or audioprocessing circuitry 318 of the sink device 300. A hard drive, CD-ROM,and RAM are some examples of articles including a non-transitorycomputer-readable medium such as a storage device. The terms“computer-readable medium” and “storage device” do not include carrierwaves to the extent carrier waves are deemed too transitory.

As described below with reference to FIGS. 6-9B, the HDMI receiver 311extracts SDR metadata from AVI InfoFrames, S-HDR metadata from DRAMInfoFrames, and/or dynamic HDR metadata from HDR DME InfoFrames in thedata islands received during the FSTW and provides at least the S-HDRmetadata and dynamic HDR metadata to InfoFrame processing circuitry 314.This metadata may include, for example and without limitation, datadescribing the format of the video data (e.g., the number of bits or thecolor configuration) and data describing an EOTF to be applied to thevideo data prior to display.

The communication interface 312 of the HDMI receiving unit 310 iscontrolled by the processor 302 to communicate with the source device200 via the DDC/SDA/SCL channel of the HDMI interface. The processor 202uses this interface to receive commands and data from, and to transmitcommands and data to, the source device 200 via the communicationinterface 312. For example, the sink device 300 may provide to thesource device 200 information (e.g., a vendor-specific data block(VSDB)) indicating the capabilities of the sink device 300 Similarly,the sink device 300 may obtain information about the source device 200via the DDC/SDA/SCL channel of the HDMI interface.

FIG. 4 is a timing diagram showing a sequence of video fields/frames 400including frame-synchronous transmission windows (FSTWs) 414 and 452according to an example embodiment. The example sequence offields/frames 400 includes two video fields/frames 410 and 450. As shownwith reference to field/frame 410, each field/frame includes a verticalblanking interval (VBI) 412, a horizontal blanking interval (HBI) 416,and an active video area 420. The vertical blanking interval 412includes a FSTW 414 and a non-FSTW region 418. The FSTW 414 begins onthe first blank pixel that immediately follows the last active videopixel of a video frame/field and ends FAPA_end lines prior to the startof the next active video area 420, where FAPA_end is defined in section10.10.1.1 of the HDMI 2.1 technical standard. Control data sent duringthe FSTW 414 is applied to the active video data in the active videoarea 420 immediately following the VBI 412 in which the FSTW 414 occurs.The control information may include, without limitation, auxiliary videoinformation (AVI) InfoFrames (AVI IFs), DRange and Mastering (DRAM)InfoFrames (DRAM IFs) and HDR DME InfoFrames (HDR DME IFs).

In sink devices that support frame-synchronous processing, controlinformation in the HDR DME is applied to the immediately followingactive video data so dynamic HDR video data in the active video area 420is properly displayed. Sink devices supporting frame synchronousprocessing identify the HDR DME and copy metadata data to appropriatecontrol registers and memory elements in the sink device 300. This mayinclude, for example, copying EOTF data to implement a particular EOTFto be used for displaying the dynamic HDR video data or configuring thesink device 300 to handle the pixel depth (e.g., the number of bits ineach pixel) or a particular color space configuration indicated by theHDR DME.

The example sink device 300 includes a vendor-specific data block (VSDB)(not shown), for example in the E-EDID, containing information on thecapabilities of the sink device 300. The VSDB may indicate that the sinkdevice 300 supports only SDR video data; SDR and S-HDR video data; orSDR, S-HDR, and dynamic HDR video data. As described above, when thesink device 300 does not support either dynamic HDR video data or S-HDRdata, the source device may convert the dynamic HDR data to S-HDR datacompatible with the AVI InfoFrames, and may convert the S-HDR data toSDR data compatible with the AVI InfoFrames before sending the convertedvideo data to the sink device 300. The example embodiments send the AVIInfoFrames, DRAM InfoFrames, and HDR DME InfoFrames during the region ofthe vertical blanking interval beginning at the first blank pixel thatimmediately follows the last active video pixel of a video frame/fieldand ending FAPA_end lines prior to the start of the next active region.This region corresponds to the FSTW 414 described above.

FIG. 5 is a timing diagram showing a single video 720P field/frame 500according to an example embodiment. The field/frame 500 shown in FIG. 5includes a horizontal sync pulse 502, a vertical sync pulse 504, a VBI506, a horizontal blanking interval (HBI) 508, and an active pixel area510. The VBI 506 includes the FSTW 512. Also included in the VBI 506 andHBI 508 are multiple data islands 514. As described above, audio dataassociated with the video data in the active pixel area 510 is sent inVBI 506 and HBI 508. The audio data and other auxiliary data may be sentin data islands 514 of the VBI 506 and HBI 508. As described above, inthe example embodiment, video metadata, including AVI InfoFrames, DRAMInfoFrames, and HDR DME InfoFrames are also sent in data islands 514,but during the FSTW 512.

FIG. 6 is a timing diagram for an existing HDMI system conforming toHDMI 2.0. The timing diagram shows a video sequence 600 havingtransitions between static high dynamic range (S-HDR) and standarddynamic range (SDR) video fields/frames. FIG. 6 illustrates artifactsthat may occur on transitions from S-HDR video frames to SDR videoframes. FIG. 6 illustrates an expected flow of video information and anactual flow showing image artifacts resulting from mismatches betweenthe video data and the metadata used to process the video data. Theexpected flow includes a first SDR sequence 602 of video fields orframes followed by a first sequence 604 of S-HDR video fields or frames.The sequence 604 is followed by a second sequence 606 of SDRfields/frames, a second sequence 608 of S-HDR video fields/frames, and athird SDR sequence 610 of video fields/frames.

Metadata for the SDR and S-HDR video data is contained in AVIInfoFrames. Although FIG. 6 shows the AVI InfoFrames and DRAM InfoFramesspanning multiple frame times, the AVI InfoFrames and DRAM InfoFramesmay be received during each field/frame interval or during alternatefields/frame intervals. Thus, data in an AVI InfoFrame or DRAM InfoFramemay be applied to one video field/frame or to two consecutive videofields/frames. The metadata for the first SDR sequence 602 is containedin a first AVI InfoFrame 612 which, as shown in FIG. 6, is received infield/frame time T1 and is active over field/frame times T1 to T100. AtT100, the AVI InfoFrame 614 for the first S-HDR sequence 604 is receivedand is active over fields/frames T100 to T301. At field/frame time T300,the sink device 300 receives the second AVI InfoFrame 616 correspondingto the field/frame sequence 606 having the second SDR sequence. As shownin FIG. 6, however, the data in the InfoFrame 616 does not become activeuntil field/frame time T301. At field/frame time T500, AVI InfoFrame 618for the second S-HDR sequence 608 is received and is active fromfield/frame time T500 to field/frame time T601. At field/frame timeT600, the sink device 300 receives the AVI InfoFrame 620 for the thirdSDR sequence 610 which does not become active until field/frame timeT601.

The metadata for the S-HDR video sequence 604 is contained in an AVIInfoFrame 614 and in DRAM InfoFrame 622, which are transmitted by thesource device 200 at field/frame time T100 but do not become activeuntil field/frame time T101. Similarly, at field/frame time T500, thesource device 200 sends the second S-HDR metadata in AVI InfoFrame 618and DRAM InfoFrame 624. The metadata in these two InfoFrames 618, 624becomes active at field/frame time T501 and remains active until timeT601, when the metadata in the AVI InfoFrame 620 for the third SDRsequence 610 becomes active.

In FIG. 6, the first displayed SDR sequence 626 is presented betweenfield/frame times T0 and T100 and the first displayed S-HDR sequence 630is presented between field/frame times T100 and T300. Betweenfield/frame times T300 and T500, the second displayed SDR sequence 634is presented; between field/frame times T500 and T600, the seconddisplayed S-HDR sequence 638 is presented; and after field/frame timeT600, the third displayed SDR sequence 642 is presented.

As shown in FIG. 6, the actual flow of the video data may exhibitartifacts caused by a mismatch between the video data being processedand the metadata used to process it. This mismatch occurs, for example,in the video field/frame 628 displayed between field/frame times T100and T101, video field/frame 632 displayed between field/frame times T300and T301, video field/frame 636 displayed between field/frame times T500and T501, and video field/frame 640 displayed between field/frame timesT600 and T601. These artifacts occur because the video data beingdisplayed in these intervals is processed using metadata correspondingto the video data from prior fields/frames. For example, the artifactoccurring in the video field/frame 636 between field/frame times T500and T501 occurs because S-HDR video sequence 638 is interpreted withoutusing the S-HDR metadata contained in the AVI InfoFrame 618 and DRAMInfoFrame 624. This distortion may be manifest as incorrect dimming withmissing shadow details and possibly incorrect color rendering. It isnoted that this distortion may be relatively insignificant, occupying asingle field/frame between longer segments of video field/frames, forexample at the beginning and/or end of a television program.

FIG. 7 is a timing diagram of a video sequence 700 showing transitionsamong SDR, S-HDR, and dynamic HDR video frames that conform to the HDMI2.1 standard. In addition to SDR and S-HDR video sequences, the exampleshown in FIG. 7 includes two video sequences having dynamic HDR videodata. The metadata, HDR DME, associated with the dynamic HDR video datafor a particular set of fields/frames is sent in AVI InfoFrames and HDRDME InfoFrames during the FAPA area of a VBI and is processed in aframe-synchronous manner, such that it is applied to the video dataoccurring immediately after the VBI. Because the HDR DME InfoFrames areapplied frame-synchronously, the HDR DME InfoFrames are sent in the FAPAregion preceding the active video interval of each dynamic HDR frame.The metadata for the SDR video and S-HDR video is included in dataislands in the VBI that are outside of the FAPA area. The expected videoflow in FIG. 7 includes a first SDR video sequence 702 betweenfield/frame times T0 and T100; a first S-HDR video sequence 704 betweenfield/frame times T100 and T200; a first dynamic HDR video sequence 706between field/frame times T200 and T300; a second SDR video sequence 708between field/frame times T300 and T400; a second dynamic HDR videosequence 710 between field/frame times T400 and T500; a second S-HDRvideo sequence 712 between field/frame times T500 and T600; and a thirdSDR video sequence 714 after field/frame time T600.

The metadata for the SDR video is contained in AVI InfoFrames. The AVIInfoFrame 716 containing metadata or the first SDR video sequence 702 isreceived in data islands during the non-FAPA area of the VBI or duringthe HBI of the field/frame starting at field/frame time T0. As shown inFIG. 7, however, this metadata is not available for use by the videoprocessing circuitry 316 (shown in FIG. 3) until field/frame time T1.The first SDR AVI InfoFrame 716 metadata is active between field/frametimes T1 and T100. At frame/field time T100, metadata for the firstS-HDR video sequence 704 is received during the non-FAPA area of the VBIor during the HBI. This metadata includes AVI InfoFrame 718 and DRAMInfoFrame 730. As shown in FIG. 7, however, this metadata does notbecome active until field/frame time T101. The metadata for the firstS-HDR video sequence 704 is active between field/frame times T101 toT200.

At field/frame time T200, the sink receives the first dynamic HDR videosequence 706 and accompanying metadata including AVI InfoFrame 720 andHDR DME 734. The AVI InfoFrame 720 is received outside of the FAPAinterval of the VBI while the HDR DME is received during the FAPAinterval (FAPA0 or FAPA1) of the VBI. As shown in FIG. 7, because it isreceived during the FAPA interval, the HDR DME 734 is processed in aframe-synchronous manner and becomes active at field/frame time T200, asindicated by the arrow 732, while the AVI InfoFrame 720 does not becomeactive until field/frame time 201. The dynamic HDR metadata remainsactive until field/frame time T300.

At field/frame time T300, the sink receives the second SDR videosequence 708 and the AVI InfoFrame 722 containing the metadata for thesecond SDR sequence 708. Because the AVI InfoFrame 722 is receivedoutside of the FAPA area of the VBI, it does not become active untilfield/frame time T301 and remains active until field/frame time T400.

At field/frame time T400, the sink receives the second dynamic HDR videosequence 710 and accompanying metadata including AVI InfoFrame 724 andHDR DME 738. As shown in FIG. 7, AVI InfoFrame 724, which was receivedoutside of the FAPA area of the VBI, does not become active until afterfield/frame time T400 while HDR DME 738, which is received during theFAPA area, becomes active at field/frame time T400 as indicated by arrow736. The dynamic HDR metadata remains active until field/frame timeT500.

The sink receives the second S-HDR video sequence 712 and accompanyingmetadata at field/frame time T500. The S-HDR metadata includes AVIInfoFrame 726 and DRAM InfoFrame 740. Both of these frames are receivedoutside of the FAPA area of the VBI and, thus, do not become activeuntil field/frame time T501. The metadata for the second S-HDR videosequence 712 remains active between field/frame times T501 and T601.

At time T601, the sink receives the third SDR video sequence 714 and itsaccompanying metadata, AVI InfoFrame 728. Because the AVI InfoFrame 728is received outside of the FAPA area, it does not become active untilfield/frame time T601.

The actual flow includes several instances of mismatch between thedisplayed video data and the dynamic range metadata used to process thevideo data. For example, the display begins with displayed SDR videosequence 742 at field/frame time T100 followed by a mismatch interval744 between field/frame times T100 and T101. This mismatch occursbecause the first S-HDR video sequence 704 is processed using the SDRmetadata because the metadata in the AVI InfoFrame 718 and DRAMInfoFrame 730 for the S-HDR video sequence 704 have not been transferredto the InfoFrame processing circuitry 314 (e.g., have not become active)until field/frame time T101. From field/frame time T101 to T200, theS-HDR video data 746 is properly displayed using the first S-HDRmetadata. Even though the DRAM InfoFrame 730 metadata is active untilfield/frame time 201, there is no mismatch at the transition beginningat field/frame time T200 because the HDR DME 734 metadata overrides theDRAM InfoFrame 730 metadata. Because it is received during the FAPA0interval, the first HDR DME 734 metadata is processed in aframe-synchronous manner and is transferred to the InfoFrame processingcircuitry 314 so that the metadata may be passed to the video processingcircuitry 316 in time to process the video data at field/frame timeT200. The displayed dynamic HDR video sequence 748 continues tofield/frame time T300 at which there is another mismatch 750. Atfield/frame time T300, the first HDR DME metadata 734 is no longeractive; however, the second SDR metadata has not yet become active. Themismatch 750 occurs because the SDR video information in the field/framestarting at time T300 is processed using the AVI InfoFrame 720 metadata.Once SDR metadata in AVI InfoFrame 722 becomes active at field/frametime T301, the system properly displays the SDR video data 752 untilfield/frame time T400. At T400, again due to the frame-synchronousprocessing, the system properly displays the dynamic HDR video data 754using the second HDR DME 738 metadata and AVI InfoFrame 724. A mismatch756 occurs, however, in the field/frame starting at T500 because thesecond S-HDR metadata in DRAM InfoFrame 740 has not become active, sothat the corresponding S-HDR video data is processed using the metadatain the AVI InfoFrame 724 for the second dynamic HDR video sequence. Oncethe metadata in the AVI InfoFrame 726 and the DRAM InfoFrame 740 becomeactive at T501, the second S-HDR video data 758 is displayed properly.The actual flow continues at field/frame time T600 with another mismatch760, when the third SDR video sequence 714 is processed using the secondS-HDR metadata contained in the InfoFrames 726 and 740. The SDR videodata 762 displays properly after field/frame time T601.

Although the examples in FIGS. 6 and 7 show a delay of one frame/fieldtime for the activation of metadata from an AVI InfoFrame or a DRAMInfoFrame, it is contemplated that there may be longer delays, forexample, four or more field/frame times. These longer delays may resultin more visible artifacts.

The visual artifacts that occur on switching to SDR or S-HDR fromdynamic HDR may be more noticeable than those which occur on switchingbetween SDR and S-HDR because, due to the dynamic nature of dynamic HDRmetadata, the changes may be less predictable, unlike legacy HDMI inwhich the changes are ‘static’ or ‘pseudo-static.’ The HDMI 2.1Specification implements frame-accuracy for switching on HDR DMEprocessing but not for switching off HDR DME processing. The visualartifacts experienced during the mismatch intervals may include reducedcontrast, for mismatch interval 744, when S-HDR video is incorrectlyinterpreted as SDR video, or incorrect dimming with missing shadowdetails for mismatch 760, when SDR video is incorrectly interpreted asS-HDR video. The artifacts may also include incorrect color. Theoccurrence of these artifacts may be increased in systems operatingaccording to the HDMI 2.1 standard due to the addition of dynamic HDRsequences, since the dynamic HDR sequences may be stitched with S-HDR orSDR in a linear stream before delivery, resulting in more frequent andmore visible artifacts.

FIG. 8 is a timing diagram showing transitions among SDR, S-HDR, anddynamic HDR video sequences according to an example embodiment. In theembodiment shown in FIG. 8, all transitions among SDR, S-HDR and dynamicHDR are frame-synchronous transitions. This may be achieved, forexample, because the sink device is capable of frame-synchronousprocessing and the metadata for the SDR, S-HDR, and dynamic HDR videosequences is received in the frame-synchronous transmission window(FSTW). As described above, the FSTW begins on the first video blankpixel that immediately follows the last active video pixel of a videoframe/field and ends FAPA_end lines prior to the start of the nextactive region (as described in section 10.10.1.1 of the High-DefinitionMultimedia Interface Specification Version 2.1). When the sink deviceimplements frame-synchronous processing, the example source device sendsthe metadata during the VBI in an area corresponding to the FSTW. Whenthe sink device does not implement frame-synchronous processing, themetadata may be sent in data islands anywhere in the VBI and/or HBI andit will be handled as described above and activated with a delay of oneto four field/frame times.

FIG. 8 shows a sequence of video data including a first SDR videosequence 802, a first S-HDR video sequence 804, a first dynamic HDRvideo sequence 806, a second SDR video sequence 808, a second dynamicHDR video sequence 810, a second S-HDR video sequence 812, and a thirdSDR video sequence 814. The first SDR video sequence 802 includesmetadata defined in AVI InfoFrames 816. The SDR metadata also includesSDR metadata for the second SDR video sequence 808 in AVI InfoFrames822. SDR metadata for the third SDR video sequence 814 is contained inAVI InfoFrames 828. Metadata for the first S-HDR video sequence 804 isin AVI InfoFrames 818 and DRAM InfoFrame 830, while metadata for thesecond S-HDR video sequence 812 is in AVI InfoFrame 826 and DRAMInfoFrame 832. Metadata for the first dynamic HDR video sequence 806 isin AVI InfoFrame 820, DRAM InfoFrame 830, and HDR DME InfoFrame 834,while metadata for the second dynamic HDR video sequence 810 iscontained in AVI InfoFrame 824 and HDR DME InfoFrame 836.

As shown in FIG. 8, dynamic HDR video may use metadata from AVIInfoFrames and HDR DME InfoFrames. Optionally, dynamic HDR video mayalso use metadata from DRAM InfoFrames, as shown by the DRAM InfoFrame830. The source device 200 determines the use case for each videosequence and formats the metadata appropriately. The metadata used by aparticular video sequence may be determined from packet headers in thecorresponding InfoFrames.

In the example shown in FIG. 8, all of the AVI InfoFrames 816, 818, 820,822, 824, 826, and 828; the DRAM InfoFrames 830 and 832; and the HDR DMEInfoFrames 834 and 836 are handled in a frame-synchronous manner. Thus,as shown in the actual flow, the video data is processed using itscorresponding metadata and there are no mismatches. The sequence inwhich the video data is displayed includes the displayed SDR videosequence 838 followed by the displayed S-HDR video sequence 840, thedisplayed dynamic HDR video sequence 842, the displayed SDR videosequence 844, the displayed dynamic HDR video sequence 846, thedisplayed S-HDR video sequence 848, and the displayed SDR video sequence850.

To minimize visual artifacts in sinks that do not supportframe-accuracy, the source device 200 sends the data to the sink device300 according to the legacy HDMI standards so that all video packets areaccurately processed within a set amount of time, for example, one tofour fields/frames times after each video transition.

FIG. 9A is a flowchart diagram useful for describing the operation ofthe source device 200 capable of processing metadata forframe-synchronous operation according to an example embodiment. At block902 of a process 900, when the source device 200 is powered on, thesource device 200 obtains the VSDB from the sink device 300 to which itis connected using the communication interface 212 of the HDMI cable 140or 182, described above with reference to FIGS. 1A, 1B and 2. If, atblock 904, the source device 200 determines that the sink device 300cannot process dynamic HDR video sequences, then the source device 200,at block 906, inhibits transmission of any HDR DME InfoFrames. For thesesink devices, AVI InfoFrame metadata and DRAM metadata is transmittedduring an interval of the VBI corresponding to the FSTW.

When, at block 904, the source device 200 determines that the sinkdevice 300 can process dynamic HDR video sequences, the source device200, at block 908, formats the video data so that all of the metadata inthe AVI InfoFrames, DRAM InfoFrames, and HDR DME InfoFrames is sentduring the FSTW.

FIG. 9B is a flowchart diagram useful for describing the operation ofthe sink device 300 capable of frame-synchronous processing according toan example embodiment. At block 952 of a process 950, the sink device300 obtains the video sequences which may include SDR, S-HDR, anddynamic HDR video sequences. At block 954, the sink device 300 extractsthe metadata from AVI InfoFrames, DRAM InfoFrames, and/or HDR DMEInfoFrames received during the FSTW. At block 956, the process 950applies the metadata to the active video immediately following the VBIcontaining the FSTW. Thus, all video sequence metadata is processed in aframe-synchronous manner, whether it is metadata for an SDR sequence, anS-HDR sequence, or a dynamic HDR sequence. Although not shown in FIG.9B, when the sink device 300 receives video from a source device that isnot compatible with frame-synchronous processing, it processes the videoin the same way as a legacy device (e.g., a device operating accordingto the HDMI 2.0 or HDMI 1.4 standard).

The metadata describes how the video data sent during the active videointerval is to be displayed. For example, the metadata may include:information on color remapping; a color volume transform to be applied;maximum, minimum and average luminance values in a scene and targetmaximum, minimum and average luminance values; data describing atransfer function (e.g. an EOTF) to be applied to the luminance data;and/or data specific to an application running on the source device. Thecontent and format of the metadata is described in a standard issued bythe Consumer Technology Association™, entitled A DTV Profile forUncompressed High Speed Digital Interfaces CTA-861-G (November 2016).

With reference to FIG. 3, the HDMI receiver applies the metadata byextracting it from the received data and passing the extracted metadatato the InfoFrame processing circuitry 314. The InfoFrame processingcircuitry, in turn, controls the video processing circuitry 316 so thatthe video data received during the active video interval is displayedproperly on the sink device. The example InfoFrame processing circuitry314 processes the metadata extracted from the FSTW and controls thevideo processing circuitry 316 to apply the extracted metadata to theactive video data in the same field/frame as the FSTW.

FIG. 10 is a state diagram 1000 useful for describing differences amongthe example HDMI embodiments, HDMI 2.1, and legacy HDMI. The statediagram 1000 includes three states: SDR state 1010, S-HDR state 1012,and dynamic HDR state 1014. These states 1010, 1012, 1014 represent thesource device 200 transmitting, and the sink device 300 receiving anddisplaying, SDR video, S-HDR video, and dynamic HDR video, respectively.The diagonal line 1030 divides legacy HDMI on the left from HDMI 2.1 andthe example HDMI embodiments on the right. The other lines in FIG. 10indicate frame-synchronous metadata transitions. The dashed lines 1016,1018, and 1020 and the text that is not underlined show theframe-synchronous metadata transitions that occur in HDMI 2.1 while allof the lines 1016, 1018, 1020, 1034, 1036, 1038, 1040, 1042, and 1044and all of the text, show the frame-synchronous metadata transitions ofthe example HDMI embodiments.

As described above, with reference to FIG. 7, according to the HDMI 2.1standard, all metadata transitions to dynamic HDR are frame-synchronous.These includes the metadata transition (line 1018) from the SDR state1010 to the dynamic HDR state 1014, the metadata transition (line 1016)from the S-HDR state 1012 to the dynamic HDR state 1014, and themetadata transition (line 1020) from one set of dynamic HDR metadata toanother set of dynamic HDR metadata within the dynamic HDR state 1014.The text that is not underlined indicates the frame-synchronoustransitions that occur in HDMI 2.1. As shown, each metadata transitionmay include multiple types of InfoFrames. All states, SDR state 1010,S-HDR state 1012, and dynamic HDR state 1014, use AVI InfoFramemetadata. SDR state 1010 optionally includes metadata in DRAM InfoFramesin addition to metadata in AVI InfoFrames, for example when the SDRvideo is generated from a Blue-Ray disc. This is indicated by theparenthetical (DRAMIF) next to lines 1034, 1036, and 1042 in FIG. 10.Furthermore, while Dynamic HDR always uses metadata from HDR DMEInfoFrames and AVI InfoFrames, it may also use DRAM metadata asindicated by the parenthetical (DRAMIF) next to lines 1016 and 1020.

As shown in FIG. 8, according to the example embodiments, all metadatatransitions may be frame-synchronous. These include the HDR DMEInfoFrames and AVI InfoFrames in the transition (line 1018) from the SDRstate 1010 to the dynamic HDR state 1014; and the HDR DME InfoFrames inthe transition (line 1020) within the dynamic HDR 1014 state. As shownby the underlined text adjacent to line 1020, in some embodiments,transitions of metadata in the AVI InfoFrames and DRAM InfoFrames withinthe dynamic HDR state 1014 may also be frame synchronous. As shown byline 1034, transitions of metadata in AVI InfoFrames and optionally DRAMInfoFrames from the dynamic HDR state 1014 to the SDR state 1010 may beframe-synchronous as may be the transitions (line 1036) of metadata inAVI InfoFrames and optionally DRAM InfoFrames within the SDR state 1010.As shown by line 1038, transitions of metadata in AVI InfoFrames andDRAM InfoFrames from the SDR state 1010 to S-HDR state 1012 may beframe-synchronous, as may the transitions of metadata in AVI InfoFramesand DRAM InfoFrames within the S-HDR state 1012, as indicated by line1040. Line 1042 shows the transition from the S-HDR state 1012 to theSDR state 1010. The AVI InfoFrame metadata and optionally the DRAMInfoFrame metadata transitions for the transition indicated by line 1042may be frame-synchronous. Finally, as shown by line 1044, AVI InfoFrameand DRAM InfoFrame metadata transitions from the dynamic HDR state 1014to the S-HDR state 1012 may be frame-synchronous.

Although the examples described above concern metadata transitionsrelated to the changing dynamic range of the video signals, it iscontemplated that other metadata transitions in video or audio signalsmay be implemented as frame-synchronous transitions. For example,object-oriented audio and video data such as may be used invirtual-reality and augmented-reality applications may be transmittedthrough the HDMI interface. In this instance, frame-synchronousprocessing may be desirable to coordinate the video and audio data tomotions and/or gestures of the user.

1. (canceled)
 2. An apparatus for use in a source device fortransmitting data using a high definition media interface (HDMI), theapparatus comprising: an HDMI interface for transmitting data to andreceiving data from a sink device; a memory holding executable code; anda processor, coupled to the memory and to the HDMI interface, theprocessor configured by the executable code to perform operationsincluding: obtaining video data and metadata for transmission to thesink device; and transmitting the metadata during a frame-synchronoustransmission window (FSTW) of the video data, wherein the FSTW beginsduring a video blanking interval (VBI) of the video data.
 3. Theapparatus of claim 2, wherein the FSTW begins on a first video blankpixel of a vertical blanking interval of a video field or video frame ofthe video data that immediately follows a last active video pixel of apreceding video frame or video field and ends a predetermined number ofvideo lines after a start of the VBI.
 4. The apparatus of claim 2,wherein the obtained metadata includes metadata for a static highdynamic range (S-HDR) video sequence and the operations further include:encoding the metadata for an S-HDR video field or S-HDR video frame ofthe S-HDR video sequence in an auxiliary video information (AVI)information frame and in a DRange and Mastering (DRAM) informationframe; and transmitting the AVI information frame and the DRAMinformation frame during the FSTW of the S-HDR video field or S-HDRvideo frame.
 5. The apparatus of claim 2, wherein the obtained metadataincludes metadata for a dynamic high dynamic range (HDR) video sequenceand the operations further include: encoding the metadata for a dynamicHDR video frame or dynamic HDR video field of the dynamic HDR videosequence in an auxiliary video information (AVI) information frame andin a HDR dynamic metadata extended (HDR DME) information frame; andtransmitting the AVI information frame and the HDR DME information frameduring the FSTW of the dynamic HDR video field or dynamic HDR videoframe.
 6. The apparatus of claim 2, wherein the operation oftransmitting the metadata via the HDMI interface further includesencoding the metadata in an auxiliary video information (AVI)information frame.
 7. The apparatus of claim 2, wherein: the operationof obtaining the video data and the metadata for transmission to thesink device further includes obtaining video data and metadata for astatic high dynamic range (S-HDR) video sequence and obtaining videodata and metadata for a dynamic high dynamic range (HDR) video sequence;and the operation of transmitting the metadata via the HDMI interfacefurther includes: transmitting the metadata for the S-HDR video sequenceduring the FSTW of a first S-HDR video field or S-HDR video frame of theS-HDR video sequence; and transmitting the metadata for the dynamic HDRvideo sequence during the FSTW a first dynamic HDR video field ordynamic HDR video frame of the dynamic HDR video sequence.
 8. Theapparatus of claim 7, wherein: the operation of obtaining the video dataand the metadata for transmission to the sink device further includesobtaining video data and metadata for standard dynamic range (SDR) videosequence; and the operation of transmitting the metadata via the HDMIinterface further includes: transmitting the metadata for the SDR videosequence during the FSTW of a first SDR video field or SDR video frameof the SDR video sequence.
 9. The apparatus of claim 2, wherein obtainedvideo data includes standard dynamic range (SDR) video data and themetadata includes metadata for the SDR video data.
 10. An apparatus foruse in a sink device for receiving data using a high-definition mediainterface (HDMI), the apparatus comprising: an HDMI interface forreceiving data from a source device; a memory holding executable code;and a processor, coupled to the memory and to the HDMI interface, theprocessor configured by the executable code to perform operationsincluding: receiving a video sequence from the source device via theHDMI interface, the video sequence including a plurality of video fieldsor video frames, each video field or video frame including an activevideo interval and one or more video blanking intervals (VBIs);extracting metadata for the video sequence from a frame-synchronoustransmission window (FSTW) one video field or video frame of the videosequence, wherein the FSTW begins during one VBI of the one or moreVBIs, on a first video blank pixel that immediately follows a lastactive video pixel of a preceding video frame or video field and ends apredetermined number of video lines after a start of the one VBI; andapplying the extracted metadata to video data in the active videointerval of the one video field or video frame.
 11. The apparatus ofclaim 10, wherein the received video sequence includes a static highdynamic range (S-HDR) video sequence including a plurality of S-HDRvideo fields or S-HDR video frames and the operations further include:extracting a DRange and Mastering (DRAM) information frame from at leastone FSTW of at least one S-HDR video field or S-HDR video frame of theplurality of S-HDR video fields or S-HDR video frames: extracting themetadata from the DRAM information frame; and applying the extractedmetadata to the video data in the active video interval of the one S-HDRvideo field or S-HDR video frame.
 12. The apparatus of claim 10, whereinthe received video sequence includes a dynamic high dynamic range (HDR)video sequence and the operations further include: extracting a dynamicHDR dynamic metadata extended (HDR DME) information frame from at leastone FSTW of one dynamic HDR video field or dynamic HDR video frame ofthe HDR video sequence; extracting the metadata from the HDR DMEinformation frame; and applying the extracted metadata to the video datain the active video interval of the one dynamic HDR video field ordynamic HDR video frame.
 13. The apparatus of claim 10, wherein: theoperation of receiving the video sequence includes receiving a statichigh dynamic range (S-HDR) video sequence and receiving a dynamic highdynamic range (HDR) video sequence; and the operation of extracting themetadata from the FSTW further includes: extracting the metadata for theS-HDR video sequence from the FSTW of a first S-HDR video field or S-HDRvideo frame of the S-HDR video sequence; and extracting the metadata forthe dynamic HDR video sequence from the FSTW of a first dynamic HDRvideo field or dynamic HDR video frame of the dynamic HDR videosequence.
 14. the apparatus of claim 10, wherein the operation ofextracting the metadata from the video sequence further includesextracting the metadata from an auxiliary video information (AVI)information frame of the video sequence.
 15. The apparatus of claim 10,wherein video sequence includes a standard dynamic range (SDR) videosequence and the operation of extracting the metadata from the videosequence further includes extracting metadata for the SDR videosequence.
 16. A method for processing data from a high-definition mediainterface (HDMI), the method comprising: obtaining a video sequence viaan HDMI interface, the video sequence including a plurality of videofields or video frames, each video field or video frame including anactive video interval and one or more video blanking intervals (VBIs);extracting metadata for the video sequence from a frame-synchronoustransmission window (FSTW) of one video field or video frame of thevideo sequence, wherein the FSTW begins during one VBI of the one ormore VBIs of the video data, on a first video blank pixel thatimmediately follows a last active video pixel of a preceding video frameor video field and ends a predetermined number of video lines after astart of the one VBI; and applying the extracted metadata to video datain the active video interval of the one video field or video frame. 17.The method of claim 16, wherein the video sequence includes a statichigh dynamic range (S-HDR) video sequence including a plurality of S-HDRvideo fields or S-HDR video frames and the method further comprises:extracting a DRange and Mastering (DRAM) information frame from at leastone FSTW of at least one S-HDR video field or S-HDR video frame of theplurality of S-HDR video fields or S-HDR video frames; extracting themetadata from the DRAM information frame; and applying the extractedmetadata to the video data in the active video interval of the one S-HDRvideo field or S-HDR video frame.
 18. The method of claim 16, whereinthe video sequence includes a dynamic high dynamic range (HDR) videosequence and the method further comprises: extracting a dynamic HDRdynamic metadata extended (HDR DME) information frame from at least FSTWof at least one dynamic HDR video field or dynamic HDR video frame ofthe HDR video sequence; extracting the metadata from the HDR DMEinformation frame; and applying the extracted metadata to the video datain the active video interval of the one dynamic HDR video field ordynamic HDR video frame.
 19. The method of claim 16, wherein: theobtaining of the video sequence includes obtaining a static high dynamicrange (S-HDR) video sequence and obtaining a dynamic high dynamic range(HDR) video sequence; and the extracting of the metadata from the FSTWfurther includes: extracting the metadata for the S-HDR video sequencefrom the FSTW of a first S-HDR video field or S-HDR video frame of theS-HDR video sequence; and extracting the metadata for the dynamic HDRvideo sequence from the FSTW of a first dynamic HDR video field ordynamic HDR video frame of the dynamic HDR video sequence.
 20. themethod of claim 16, wherein the extracting of the metadata from thevideo sequence further includes extracting the metadata from anauxiliary video information (AVI) information frame of the videosequence.
 21. The method of claim 16, wherein video sequence includes astandard dynamic range (SDR) video sequence and the extracting of themetadata from the video sequence further includes extracting metadatafor the SDR video sequence.